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EXECUTIVE  SUMMARY 


DoD  and  the  Services  have  shown  an  increasing  awareness  that 
there  is  an  uncertainty  associated  with  Life  Cycle  Cost  Estimates  (LCCE) 
and  that  the  further  system  costs  are  projected  into  the  future,  the 
greater  is  the  uncertainty  of  those  costs.  Current  systems  acquisition 
policy  documents  (DoDD  5000.1  and  implementing  instructions)  emphasize 
the  need  for  displaying  and  considering  this  uncertainty  at  decision- 
making points.  These  documents  do  not,  however,  specify  techniques 
to  be  used  in  the  display  and  evaluation  of  cost  uncertainty. 

This  Study  Report  develops  a simple,  statistically  valid, 
methodology  for  evaluating  and  displaying  the  uncertainty  associated 
with  cost  estimates,  in  general,  and  with  LCCE's,  in  particular.  This 
uncertainty  can  be  displayed  as  a band  of  costs  on  either  side  of  a 
"most-likely"  cost  and  the  total  cost  can  be  displayed  within  a 
confidence  interval. 

The  technique  relies  upon  the  most  knowledgeable  individuals 
within  the  project  organization  for  basic  input  and  the  computations 
are  relatively  simple  and  easily  computerized.  The  basic  philosophy 
upon  which  the  methodology  was  developed  was  that  it  should  be  based 
on  simple  concepts  and  be  simple  to  use  and  understand. 
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PREFACE 


The  intent  of  this  paper  is  to  provide  a display 
technique  to  be  used  in  conjunction  with  U.  S.  Army 
Safeguard  System  Command  (USASAFSCOM)  Regulation  No.  11-1. 
This  regulation  prescribes  the  policies,  responsibilities, 
and  procedures  to  be  used  in  developing,  coordinating, 
validating,  presenting,  and  approving  cost  estimates 
within  USASAFSCOM.  As  presently  published,  the  reg- 
ulation has  a void  in  that  it  does  not  contain  guidance 
as  to  how  to  present  the  uncertainty  associated  with 
the  estimates  that  are  developed  in  accordance  with  the 
regulation.  The  technique  presented  is  applicable  to 
the  base  estimate  (para.  7a),  "What  If"  exercises 
(para.  7d(l)(g)),  cost  estimates  associated  with  eco- 
nomic analyses  (para  7i),  and  the  parametric  estimates 
(Appendix  D)  developed  as  specified  in  the  regulation. 

The  technique  can  also  be  used  to  display  the  estimate 
confidence  as  required  by  para.  lc(5)  of  Appendix  J and 
is  particularly  applicable  to  the  display  of  cost  un- 


technique  for  a total  cost  estimate  found  on  page  6 of 
that  appendix  to  the  dollar  streams  associated  with  a 
Life  Cycle  Cost  Estimate.  The  uncertainty  displays 
resulting  from  the  technique  presented  herein  can  be  used 
by  an  analyst  in  structuring  realistic  ranges  for  the 
significant  cost  parameters  in  performing  sensitivity 
analyses  as  required  by  Appendix  M of  the  regulation. 

The  paper  has  been  written  so  as  to  stand  alone 
since  this  was  the  philosophy  followed  in  the  prepara- 
tion of  the  regulation.  Copies  of  this  document  have 
been  forwarded  to  USASAFSCOM  for  possible  inclusion  in 
future  revisions  of  USASAFSCOM  Reg.  No.  11-1. 
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A 'IIMPLE  STATISTICAL  METHOD  OF  PRESENTING  THE  UNCERTAINTY 


ASSOCIATED  WITH  LIFE  CYCLE  COST  ESTIMATES* 

I.  Introduction 

The  task  of  developing  a Life  Cycle  Cost  Estimate  (LCCE)  for  a 
proposed  program  (or,  for  that  matter,  for  the  remainder  of  any 
program  at  a particular  point  in  its  life  cycle)  is  an  extremely 
difficult  job  at  best.  The  forecasting  of  costs  for  a system  based 
on  historical  data,  regardless  of  the  volume  or  validity  of  that 
data,  is  fraught  with  uncertainty.  It  is  precisely  this  uncertainty 
and  the  manner  in  which  it  can  be  displayed  that  is  the  subject  of 
this  report.  The  purpose  of  this  paper,  then,  is  to  develop  a 
methodology  for  displaying  cost  uncertainty  that  is  statistically 
valid,  but  based  on  simple  concepts , is  simple  to  use,  and  most 
importantly,  is  simple  to  understand. ^ 

The  primary  need  for  a cost  estimate  in  the  decision-making  process 
is  to  assist  the  decision  maker  in  forming  a basis  for  comparison 
between  two  or  more  alternatives — whether  or  not  to  acquire  a system, 
which  of  two  systems  to  acquire,  etc.  Uncertainty  analysis  can 
provide  the  decision  maker  with  information  which,  when  taken  in 
proper  context,  should  improve  the  rationality  of  his  decisions. 


*AB STAINER 

This  study  represents  the  views,  conclusions,  and  recommendations  of 
the  author  and  does  not  necessarily  reflect  the  official  opinion  of  the 
Defense  Systems  Management  School  nor  the  Department  of  Defense. 
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What  is  uncertainty  in  a cost  estimate  and  how  does  it  arise  in 
the  first  place?  From  the  decision-maker's  point  of  view,  uncertainty 
is  that  nagging  feeling  in  the  back  of  his  mind  that,  although  the 
array  of  numbers  before  him  is  impressive  in  its  seeming  completeness, 
and  although  the  numbers  in  the  rows  and  columns  add  up  as  they 
should,  something  is  missing  that  prevents  the  whole  picture  from 
being  in  focus.  That  thing  that  is  missing  is  a measure  of  the 
validity  of  the  numbers  that  the  decision  maker  has  before  him.  The 
decision-maker's  troubled  feeling  should  stay  with  him  until  he  can  i 

relate  a measure  of  confidence  to  the  numbers  he  has  in  his  possession. 

He  must  always  associate  some  measure  of  uncertainty  with  his  estimate 
and  never  fall  into  the  trap  of  considering  it  as  a fact  as  he  uses 
it  in  the  day-to-day  management  of  his  program.  If  the  uncertainty 
associated  with  LCCE's  is  not  addressed,  but  rather  is  assumed  away 

by  regarding  the  estimate  as  a concrete  statement  of  fact,  management  ’ 

] 

cost  problems  do  not  disappear,  but  rather  multiply  dramatically  and  j 

can  Include  such  things  as  annual  underfunding  of  the  program, 

repeated  deferrals  of  work  to  succeeding  years  requiring  repeated 

program  realignments,  cost  overruns,  and  the  ultimate  management 

disaster-threat  of  early  program  termination  because  of  repeated 

financial  difficulties. 

A memorandum,  dated  5 August  1970,  and  signed  by  the  then 
Deputy  Secretary  of  Defense  David  Packard,  defined  nine  mandatory 
categories  of  cost  growth,  as  follows; 

: 
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1.  Engineering  Change 
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2.  Quantity  Change 

3.  Support  Change 

4.  Schedule  Change 

5.  Unpredictable  Change 

6.  Economic  Change 

7.  Estimating  Change 

8.  Contract  Performance  Incentives 

9.  Contract  Cost  Overrun  (Underrun) 

A copy  of  the  memorandum  defining  the  above  categories  of  cost 
growth  is  contained  in  the  Appendix  to  this  paper. 

Since,  by  definition,  any  deviations  from  an  estimate  to  actual 
costs,  or  from  one  estimate  to  a succeeding  estimate,  must  be 
identified  to  one  of  the  above  categories  of  cost  growth,  they  and 
inflation  (which  can  also  be  included  in  cost  growth  as  "economic 
change")  are  the  total  source  of  uncertainty  in  a cost  estimate. 

Figure  1 illustrates  the  effect  that  the  uncertainty  associated  with 
cost  growth  and  inflation  can  have  on  actual  costs  when  compared  with 
an  LCCE  for  the  same  program.  The  uncertainty  associated  with  the 
estimate  tends  to  compound  with  time  (although  not  necessarily  as  a 
linear  function,  as  in  the  figure)  so  that  the  further  one  projects 
cost  data,  the  greater  is  the  magnitude  (in  dollars)  of  the  uncertainty 
associated  with  the  estimate.  For  example,  at  point  t in  Figure  1, 
the  originally  estimated  cumulative  cost  for  a program  may  have  been 
represented  by  point  a,  but  the  actual  inflation  experienced  and  cost 
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growth  may  cause  the  costs  to  reach  point  b,  or  any  other  point 
between  a and  b.  (It  should  also  be  noted  that  if  the  estimate  was 
unduly  pessimistic,  the  actual  cost  could  fall  below  a,  but  from  a 
management  point  of  view,  this  result  is  equally  undesirable,  since 
perishable  assets  may  have  been  committed  to  the  program  and  not 
utilized. ) 

DoD  and  the  Services  have  recognized  the  need  for  including  a 
consideration  of  the  uncertainty  associated  with  cost  estimates  as  a 
means  of  assisting  in  the  control  of  resources.  The  basic  policy  with 
respect  to  the  utilization  of  resources  in  the  acquisition  process  is 
promulgated  in  DoD  Directive  5000.1,  dated  13  July  1971.  Within  the 
Army,  a Letter  of  Instructions  (LOI) , subject:  Implementing  the 

New  Materiel  Acquisition  Guidelines,  dated  23  August  1972f  contains 
more  specific  instructions  on  the  use  of  cost  estimates.  Annex  L of 
this  LOI  states  the  following: 

. . . Cost  estimates  should  be  expressed  in  terms  of 
limits  within  which  the  ultimate  cost  of  the  program  may 
be  expected  to  fall....* 


Methodoloc 


Having  now  established  the  need  for  including  a consideration  of 
uncertainty  in  the  formulation  of  an  LCCE  and  briefly  stated  the 
possible  consequences  of  omitting  this  consideration  in  the  use  of 
these  estimates,  the  remainder  of  this  paper  will  be  devoted  to  the 
development  of  a simple  technique  that  can  be  applied  by  individuals 
with  only  a general  knowledge  of  statistics  for  establishing  and 
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displaying  uncertainty  so  that  this  display  might  serve  to  remind 
managers  that  an  estimate  is  exactly  that.  At  a particular  point  in 
the  future,  costs  are  better  thought  of  as  "ranges  of  possible  costs" 
or  "cost  bands"  and  management  decisions  should  be  based  on  that 
premise.  That,  then,  is  the  theme  that  we  follow  from  this  point 
forward.  The  proposed  technique  for  displaying  cost  uncertainty  will 
be  described  in  general  terms  as  a means  of  providing  the  rationale 
leading  to  its  development.  Following  this  general  discussion,  specific 
computational  examples  Illustrating  the  salient  points  will  be 
presented. 

Figure  2 is  a graphical  presentation  of  the  dominant  categories 
of  costs  present  in  an  LCCE.  The  total  cost  estimate  at  any  point  in 
time  will  be  the  sum  of  these  categories  at  that  point  (considering 
the  categories  to  be  all  inclusive).  Characteristically  RLT&E, 
Procurement,  and  Operating  costs  phase  in  generally  as  shown  (there 
are  certain  Operating  funds  required  early  in  the  program  to  pay  the 
salaries  of  Government  employees  who  cannot  be  charged  to  the  RDT&E 
appropriation).  These  categories  of  costs  are  widely  used  in  the 
management  process  (they  are  the  categories  by  which  Congress 
appropriates  funds)  and  form  a logical  basis  for  the  development  of  a 
Cost  Breakdown  Structure  (CBS)  to  be  used  in  the  development  of  an 
LCCE.  The  CBS  developed  should  also  be  directly  related  to  the  program 
Work  Breakdown  Structure  (WBS)  in  the  sense  that  costs  associated  with 
the  accomplishment  of  elements  of  the  WBS  should  sum  to,  or  be,  the 
elements  of  the  CBS,  l.e. , the  costs  associated  with  a specific  work 


Procurement 
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package  at  a particular  WBS  level  should  not  be  split  among  cost 
elements  at  a corresponding  level  in  the  CBS.  Table  1^ depicts  a 
typical  CBS  used  in  the  development  of  an  LCCE.  (For  the  purpose 
of  this  paper,  the  life  cycle  will  be  considered  to  be  ten  years.) 

This  CBS  is  described  in  a USAMC  handbook  of  Cost  Analysis  definitions 
(see  bibliography) . 

In  developing  the  CBS  for  an  LCCE  using  the  technique  presented 
in  this  paper,  and,  in  particular,  in  determining  how  many  levels  of 
the  CBS  one  must  penetrate  in  assembling  and  presenting  the  estimate, 
it  must  be  considered  that  at  least  30  cost  elements  must  be  present 

5 

(i.e.,  non-zero)  in  each  yearly  column  for  statistical  validity.  In 
the  third-level  of  the  typical  CBS  presented  in  Table  1,  there  are 
49  cost  elements  that  may  be  present  in  a given  year. 

The  next  step  in  the  preparation  of  the  LCCE  by  this  technique 
is  the  most  critical,  by  far.  The  individual  assembling  the  estimate 
must  contact  the  individuals  responsible  for  the  management  of  the 
work  associated  with  each  element  in  the  CBS  and  obtain  three  decisions 
associated  with  each  CBS  element.  These  decisions  are: 

1.  The  "most-likely"  cost  of  that  element  for  each  year  in 
the  life  cycle. 

2.  The  "lowest-likely"  cost  of  that  element  for  each  year 
in  the  life  cycle. 

3.  The  "highest-likely"  cost  of  that  element  for  each  year 
in  the  life  cycle. 
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Table  1 


TYPICAL  COST  BREAKDOWN  STRUCTURE 


1.0  Research  and  Development 

1.01  Contract 

1.011  Engineering* 

1.012  Tooling* 

1.013  Prototype  Production* 

1.014  Other 

1.015  General  and  Administrative 

1.016  Profit 

1.02  In-House 

1.021  Engineering* 

1.022  Tooling* 

1.023  Prototype  Production* 

1.024  Other 

2.0  Investment  Non-Recurring 

2.01  Contract 

2.011  Advanced  Production  Engineering* 

2.012  Tooling* 

2.013  Manufacturing* 

2.014  Quality  Control 

2.015  Other 

2.016  General  and  Administrative 

2.017  Profit 

2.02  In-House 

2.021  Advanced  Production  Engineering* 

2.022  Tooling* 

2.023  Other 

3.0  Investment  Recurring 

3.01  Contract 

3.011  Engineering* 

3.012  Tooling* 

3.013  Quality  Control* 

3.014  Manufacturing* 

3.015  Leasehold 

3.016  First  Destination  Transportation 

3.017  Other 

3.018  General  and  Administrative 

3.019  Profit 

3.02  In-House 

3.021  Engineering* 

3.022  Tooling* 

*Cost  Includes:  Direct  Labor,  Material,  Overhead  and  Other  Direct  Charges. 
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3.023 

Quality  Control* 

3.024 

Manufacturing* 

3.025 

Transportation 

3.026 

Other 

4.0 

Operation 

4.01 

Contract 

4.011 

Personnel 

4.012 

Consumption 

4.013 

Leasehold 

4.014 

Integrated  Logistic  Support 

4.015 

Transportation 

4.016 

Depot  Maintenance 

4.017 

Other 

4.02 

In-House 

4.021 

Personnel 

4.022 

Consumption 

4.023 

Leasehold 

4.024 

Integrated  Logistic  Support 

4.025 

Transportation 

4.026 

Depot  Maintenance 

4.027 

Other 

*Cost  Includes:  Direct  Labor,  Material,  Overhead  and  Other  Direct  Charges. 


10 


These  decisions  could  be  stated  in  another  way  as  a matter  of 
convenience  to  the  individual  making  the  decision,  i.e.,  the  "most- 
likely"  cost  and  a percentage  range  of  that  cost  from  "lowest-likely" 
to  "highest-likely."  The  cost  decision  then  would  be  stated  as, 

"The  'most- likely ' cost  for  this  element  is  X dollars  with  a range 
from  90  to  115  per  cent." 

There  is  a large  number  of  techniques  available  to  the  cost 
element  manager  to  assist  him  in  arriving  at  his  cost  decisions.  It 
is  beyond  the  scope  of  this  paper  to  address  these  techniques  since, 
in  fact,  our  requirement  is  for  the  products  of  the  decision-making 
process  only.  The  uncertainty  present  in  these  cost  decisions  naturally 
reflects  the  bias  of  the  individuals  making  the  various  decisions. 

Since  these  individuals  are  the  managers  of  the  work  assoicated  with 
the  cost  elements,  they  are  presumed  to  be  the  most  knowledgeable 
individuals  for  those  cost  elements  and  should  provide  the  "best 
estimates."^ 

Note  that  throughout  the  preceding  paragraphs  the  individual  CBS 
element  managers  were  asked  to  make  cost  "decisions"  and  not  "estimates." 
The  choice  of  this  word  was  deliberate  and  was  intended  to  convey  to 
the  reader  some  idea  of  the  intensity  of  the  reasoning  which  those 
managers  should  apply  to  the  process  of  developing  costs.  The  importance 
of  this  concept  cannot  be  overemphasized.  From  this  point  on,  the 
estimate  is  assembled  by  universally  accepted  statistical  methods.  But 
no  statistical  methods,  no  matter  how  valid,  can  assemble  a good  cost 
decision  if  the  basic  input  is  developed  in  an  offhand  manner.  It  must 
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also  be  remembered  that  this  is  the  point  at  which  the  uncertainty 
about  cost  growth  is  input  into  the  cost  decision — by  the  individual 
with  the  most  detailed  knowledge  of  the  cost  element  and  its  likelihood 
of  being  affected  by  cost  growth.  Statistical  procedures  to  follow 
can  "smooth"  the  inconsistencies  which  may  be  in  the  estimate,  but 

I 

they  cannot  Improve  the  basic  data  provided  by  the  individual  managers.  ' 

While  making  his  cost  decisions,  the  manager  must  consider  also  i 

1 

the  time  frame  in  which  the  costs  will  be  incurred.  From  Figure  1,  | 

it  is  apparent  that  his  range  of  possible  costs  (uncertainty)  should  J 

be  much  broader  for  a cost  element  in  year  10  of  the  life  cycle  than  j 

1 

for  one  in  year  2. 

The  basic  computational  techniques  that  follov’  are  based  upon  the  | 

application  of  the  beta  probability  density  function  to  the  data 

obtained  in  the  preceding  step  (See  Wilton's  "Notes  on  the  Beta  ! 

i 

Distribution,"  bibliography,  or  any  comprehensive  text  on  statistics  j 

for  a further  description  of  this  function) . The  beta  function  is  j 

chosen  for  a number  of  reasons,  including  the  following:  ■ 

i 

1.  With  proper  shaping  parameters,  the  beta  distribution 
fits  what  intuitively  seems  to  be  the  probable  distribution  of  costs. 

The  common  skewed-right  shape  of  the  beta  curve  accommodates  the 
optimism  Inherent  in  cost  estimating  wherein  the  "most-likely"  cost 
estimate  is  usually  closer  to  the  "lowest-likely"  cost  than  the 
"highest-likely"  cost. 

2.  The  beta  distribution  has  upper  and  lower  bounds.  The 
selection  of  an  unbounded  distribution  probably  would  be  of  small 
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consequence  in  a methodology  such  as  this,  but  bounds  or  constraints 
do  exist  in  the  real  world  so  it  is  appealing  to  select  a distribution 
that  reflects  these  conditions. 

3.  Accurate  approximations  are  available  for  the  mean  and 
variance  of  the  beta  distribution  that  will  greatly  simplify  calcula- 
tions to  follow. 

The  major  disadvantage  associated  with  the  use  of  the  beta 
distribution  is  that  it  assumes  that  costs  are  distributed  in  an 
unimodal  manner  (or,  in  a special  case,  along  a straight  line).  Unless 
there  are  obvious  indications  to  the  contrary  in  a particular  applica- 
tion, this  limitation  does  not  seem  to  be  overly  severe  or  to  render 
the  total  methodology  invalid,  in  general.  The  requirement  to  model 
exactly  (or  at  least  to  closely  approximate)  the  actual  cost  distribu- 
tion is  further  mitigated  since  later  in  the  methodology  the  extremely 
powerful  "Central  Limit  Theorem"  will  be  invoked  with  regard  to  the 
yearly  cost  data  (See  Murphy's  report,  bibliography,  for  a description 
of  the  rationale  supporting  this  application) . This  theorem  implies 
that  if  the  cost  elements  are  Independently  distributed  and  have  a 
finite  variance  and  arithmetic  mean,  then  the  yearly  variances  and 
means  may  be  readily  computed.  Further,  the  distributions  of  costs 
for  the  yearly  totals  approach  nearer  the  normal  distribution  as  the 
number  of  cost  elements  increases.  This  is  true  regardless  of  the 
distribution  of  the  individual  cost  elements.  (A  typical  beta 
distribution  plot  is  shown  in  Figure  3.) 
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The  independence  of  the  cost  elements  in  the  CBS  previously 
described  is  open  to  question.  In  the  development  of  this  technique, 
independence  will  be  assumed  and  the  individual  considering  its  use 
must  determine  whether  his  cost  elements  are  actually  Independent, 
or,  if  not,  whether  the  dependence  is  of  a sufficient  degree  to 
Invalidate  the  methodology  in  that  particular  situation. 

(The  true  power  of  the  Central  Limit  Theorem  is  not  evident  in 
this  discussion  since  we  have  limited  ourselves  to  cost  elements 
distributed  in  accordance  with  the  beta  function  for  the  sake  of 
simplicity.  Any  function  closely  approximating  the  actual  cost 

i 

distribution  can,  in  fact,  actually  be  used  for  a cost  element  as 
long  as  the  variance  and  arithmetic  mean  can  be  computed  and  a "most- 
likely"  cost  can  be  determined.)^ 

The  mean  and  variance  of  each  cost  element  in  each  of  the  columns  i 

(yearly  and  total  costs)  can  now  be  computed  using  the  following 
approximating  formulas: 

: A + 4M  + B 

Mean  M = g 


L 


Variance 

Where 


o2  = (^)2 


A ■ Lower  bound  of  distribution 
M = Mode  of  distribution 
B = Upper  bound  of  distribution 


Exact  expressions  for  the  mean  and  variance  of  the  beta 

8, 


distribution  are  available  but  the  approximations  given  above  are  of 
sufficient  accuracy  (and  much  simpler  to  use)  for  the  purpose  of  this 
methodology  (See  Murphy's  report,  bibliography). 
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I The  rationale  behind  the  requirement  for  the  cost  decisions  that 

[ the  manager  was  earlier  tasked  to  make  is  now  obvious.  The  "most- 

likely"  cost  becomes  M,  the  mode,  and  the  "lowest- likely"  and  "highest- 
likely"  costs  become  A and  B,  respectively.  (If  the  range  of  costs 
is  given  in  percentages,  they  can  be  easily  converted  to  dollar 
values  corresponding  to  the  "lowest-likely"  and  "highest-likely"  costs.) 

In  this  manner  we  have  provided  the  managers  with  a set  of  intuitive 
guidelines  to  follow  as  they  make  their  cost  decisions  that  do  not 
require  a depth  of  statistical  knowledge  to  understand  or  communicate. 

After  the  means  and  variances  for  each  element  are  computed, 
they  and  the  modes  are  summed  for  each  column  (i.e.,  yearly  cost  data 
and  total  estimate).  The  square  roots  of  the  sums  of  the  variances 
for  each  column  are  then  extracted,  giving  a standard  deviation  (in 
dollars)  for  each  yearly  column  and  the  total  column. 

The  Central  Limit  Theorem  is  invoked  at  this  point  in  the 
estimating  process.  By  the  use  of  this  powerful  tool,  the  distribution 

A 

of  the  column  total  costs  can  be  considered  to  be  normal.  The 
Implication  of  this  is  that  a confidence  interval  (i.e.,  90%,  95%, 

99%,  etc.)  can  be  selected  and  standardized  normal  random  variable 
tables  can  be  used  to  display  the  probable  distribution  of  costs.  The 
data  developed  to  this  point  can  then  be  displayed  in  a meaningful 
manner  as  we  will  see  in  an  example  to  follow. 

III.  Sample  Computations 

The  example  in  this  section  will  be  structured  using  the  CBS 

10 

developed  in  the  preceding  section.  To  avoid  repetitive  numerics, 

16 

1 


A 


only  typical  calculations  will  be  included.  To  simplify  the  CBS 
array,  the  index  numbers  from  Table  1 rather  than  the  descriptive 
titles  of  the  cost  elements  will  be  used.  All  cost  data  are  in 
millions  of  dollars. 

A.  Display  the  necessary  cost  data  for  each  cell  as  follows; 


Cost  Element: 

1.011 

A: 

10.1 

Life  Cycle  Year: 

3 

B: 

11.9 

M: 

10.7 

B. 

Compute  the  means 

and  variances  for  each 

cell  as 

follows 

A 

+ 4M  + 

B 10.1  + 42 

.8  + 

11.9 

Mean  M = 

6 

= 6 

- 

10.8 

9 B 

- A , 

1.8  0 

Variance  = (~ 

6 

= ( 6 )^  = 0-' 

09 

C. 

Compute  the  sums 

of  the 

means , modes , 

and 

variances  for 

column 

as  follows: 

CBS 

Year  3 

Element 

Mean 

Variance 

Mode 

1.011 

10.8 

0.09 

10.7 

1.012 

5.1 

0.48 

5.0 

1.013 

8.2 

0.69 

8.1 

4.027 

1.7 

0.01 

1.6 

Totals 

128.7 

65.61 

123.2 

D.  Compute  the  square  root  of  the  variance  for  each  column  as 
follows : 

/ 65.61  = 8.1 

This  calculation  yields  the  standard  deviation  (in  dollars) 
for  this  yearly  data. 

E.  Establish  the  confidence  level.  In  this  example,  we  will 


assume  that  a 95%  confidence  level  will  be  used. 


F.  Using  standardized  normal  random  variable  tables,  determine  the 


number  of  standard  deviations  on  either  side  of  the  mean  which  include 
95%  of  the  distribution.  In  this  case,  the  appropriate  number  is  1.96. 

G.  Multiply  the  standard  deviation  for  each  yearly  column  by  the 
number  determined  in  step  F,  as  follows: 

Year  3 

1.96  X 8.1  = 16.2 

H.  Add  and  subtract  the  number  developed  in  step  G to  the  mean 
for  each  year,  as  follows: 

M + 16.2  = 128.7  + 16.2  = 144.9 

M - 16.2  = 128.7  - 16.2  = 112.5 

I.  Plot  the  sum  of  the  modes  and  the  two  values  developed  in  step 
H above  for  each  year  (but  not  for  the  total  column)  on  a graph  as  shown 
in  Figure  4a.  The  data  for  all  years  can  then  be  added  as  shown  in 
Figure  4b.  A smooth  curve  can  then  be  drawn  through  the  upper  points, 
through  the  sums  of  the  modes,  and  through  the  lower  points  as  shown  in 
Figure  4c.  The  data  points  can  then  be  de-emphasized  as  shown  in  the 
final  chart,  4d,  which  would  be  used  as  the  display  chart.  The  only 
purpose  of  the  curves  in  Figure  4d  is  to  emphasize  the  range  of  the  un- 
certainty and  to  provide  some  visual  indication  of  the  relative  probability 
of  an  overrun  or  underrun  of  the  "most-likely"  cost.  Figure  4e  is 
another  method  of  displaying  this  uncertainty  by  the  use  of  bars.  The 
length  of  the  bar  represents  the  range  of  the  uncertainty,  the  horizontal 
hatching  represents  the  relative  probability  of  an  overrun,  and  the 
vertical  hatching  represents  the  relative  probability  of  an  underrun. 
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J.  The  curve  connecting  the  sums  of  the  inodes  is  the  "most- 
likely"  cost  curve.  (In  a one-point  estimate,  this  would  be  the  cost 
developed  for  each  year.)  The  outer  curves  of  Figure  4d  represent 
the  limits  of  a 95%  probability  (or  confidence)  band  about  the  yearly 
means. 

K.  The  data  for  the  total  cost  column  is  assumed  to  be  the 
following: 

Mean  539.0 

a 26.6 

Sum  of  modes  ("most-likely"  estimate)  525.0 

This  data  has  been  plotted  in  Figure  5.  The  95%  confidence  limits 
are  as  shown  on  that  figure.  We  also  note  that  the  sum  of  the  modes 
(which  we  have  been  calling  the  "most- likely"  estimate)  is  .53  o 
((539.0  - 525.0)  t 26.6)  to  the  left  of  the  mean.  By  the  use  of 
standardized  normal  random  variable  tables,  it  can  be  shown  that  the 
probability  of  underrunning  the  "most-likely"  estimate  is  29.9%  and 
the  complementary  probability  of  overrunning  this  estimate  is  70.1%. 

Figures  4d  (or  4e,  if  that  type  display  is  preferred)  and  5 are 
the  displays  that  would  be  used  to  present  the  uncertainty  associated 
with  the  LCCE. 

The  mode  of  any  of  the  distributions  developed  in  this  methodology 
can  be  considered  to  be  the  "most-likely"  estimate  for  a particular 
year  (or  for  the  total  estimate)  since  it  represents  the  sum  of  the 
individual  "most-likely"  estimates.  This,  in  fact,  would  be  the 
estimate  provided  in  a single-point  estimating  technique.  The  means 
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are  normal  distributed  and  represent  the  point  at  which  actual  costs 
are  equally  likely  to  overrun  as  to  underrun  since  the  column  totals 
are  normally  distributed.  This  is  demonstrated  in  Figure  5.  The 
means  are  thus  the  logical  basis  for  the  establishment  of  the  confidence 
bands  and  limits. 

IV.  Conclusions 

The  technique  developed  and  presented  in  this  paper  is  a valid 
and  effective  method  of  presenting  the  uncertainty  associated  with 
cost  estimates. 

The  technique  is  valid  for  other  types  of  cost  estimates  as  well 
as  LCCE. 

The  technique  is  simple  to  apply  and  obviously  adaptable  to 
computerization. 

The  information  that  is  displayed  is  presented  in  terms  of 
intuitively  appealing  concepts. 
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FOOTNOTES 


1.  The  development  of  any  methodology  to  display  the 
results  of  a cost  estimate  necessarily  involves  some 
discussion  of  the  procedures  used  in  developing  that 
estimate.  It  is  not  the  purpose  of  this  paper  to 
develop  an  estimating  technique  for  Life  Cycle  cost 
Estimates.  The  estimating  technique  used  in  this  paper 
for  illustration  of  the  display  methodology  is  derived 
from  the  standard  USAMICOM  three-point  estimating 
procedure  described  in  the  second-referenced  document 

in  the  attached  bibliography.  This  estimating  technique 
was  originally  developed  only  for  a total  systems  cost 
estimate.  The  concept  embodied  in  this  technique  has 
been  expanded  within  this  paper  for  use  with  a LCCE  and 
applied  to  a standard  USAMC  Cost  Breakdown  Structure 
for  Life  Cycle  Costs.  It  will  become  apparent  to  the 
reader  that  other  estimating  techniques  are  equally 
adaptable  to  the  display  methodology  developed  herein. 

The  conditions  that  these  other  estimating  techniques 
must  satisfy  are  contained  in  the  body  of  this  paper. 

2.  Since  published  as  AR  1000-1,  30  June  1972. 

3.  Language  similar  to  this  has  appeared  in  policy 
documents  for  a considerable  period  of  time.  However, 
cost  estimates  in  general,  and  LCCE ' s in  particular, 
have  not  commonly  been  expressed  in  terms  of  limits 
within  which  the  ultimate  cost  of  the  program  may  be 
expected  to  fall.  It  is  the  author's  contention  that 

the  reason  for  this  is  that  the  majority  of  the  techniques 
previously  developed  have  been  too  complex  and  time- 
consuming  to  be  easily  applied  by  the  program  manage- 
ment personnel  who  most  frequently  assemble  these  esti- 
mates. This  paper  attempts  to  solve  this  management 
problem  and  fill  an  existing  management  void  by 
extending  a simple,  previously-developed,  total  systems 
cost  estimating  technique  to  LCCE ' s . 

4.  USAMC,  Key  Cost  Analysis  Definitions,  p.  12. 

5.  Murphy,  E.  L.,jr.,  Statistical  Methods  of  Measuring 
the  Uncertainty  of  Cost  Estimates,  p.  2. 


6.  The  process  of  developing  a cost  estimate  is  often 
thought  of  as  a discrete  operation.  In  actuality  it 
is  most  frequently  an  iterative  process  - estimates  are 
continually  revised  as  new  information  becomes  avail- 
able, additional  constraints  are  imposed  upon  the 
estimate,  etc.  In  this  sense,  the  uncertainty  originally 
associated  with  a cost  element  by  the  estimator  may  be 
obscured  by  subsequent  requirements.  care  should  be 
taken  to  avoid  this  circumstance , if  possible,  or  to 
highlight  it  in  an  accompanying  narrative,  if  not.  It 
must  also  be  remembered  that  the  basic  cause  of  uncer- 
tainty in  a cost  estimate  is  not  the  result  of  the 
statistical  process  under  which  it  was  assembled  but 
rather  is  due  to  the  fact  that,  in  the  final  analysis, 
the  estimate  is  simply  a judgement  made  by  an  ordinary 
human  being.  It  is  for  the  managers  who  use  the  inform- 
ation to  decide  upon  the  validity  of  the  judgement  given 
a particular  expert's  knowledge  and  the  cost  element 
being  assessed.  See  Schlaifer,  Analysis  of  Decisions 
Under  Uncertainty.  Chapter  6,  for  a short  (17  pages) 
exposition  on  the  use  and  interpretation  of  data  gotten 
from  experts . 


7.  Murphy,  op . cit . , p.  2. 

8.  Wildon,  Notes  on  the  Beta  Distribution,  p.  2. 

9.  Murphy,  op . cit . . p.  2. 

10.  This  technique  for  calculating  the  yearly  means 
and  standard  deviations  is  developed  in  Murphy,  op.  cit. 
Since  this  is  a standard  technique  used  within  USAMC, 
its  use  has  been  extended  within  this  paper  to  apply  to 
the  dollar  streams  associated  with  LCCE's.  The  notation 
in  this  paper  follows  Murphy. 

11.  Once  the  estimate  has  been  completed  and  the  confi- 
dence interval  has  been  established,  several  important 
uses  can  be  made  of  the  result  in  addition  to  the  basic 
purpose  of  displaying  the  uncertainty.  Thresholds  can 
be  established  by  selecting  the  appropriate  probability 

2C  2 


that  one  is  willing  to  accept  of  their  being  exceeded 
and  determining  the  corresponding  dollar  value  from 
the  display.  The  probability  of  exceeding  an  already 
established  threshold  can  also  be  determined  by  using 
the  standardized  normal  random  variable  tables  to 
determine  the  probability  associated  with  exceeding  a 
particular  dollar  value. 
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APPENDIX 


COST  GROWTH  DEFINITIONS 


1.  General.  This  appendix  contains  the  text  of  the  Deputy  Secretary 

of  Defense  Memorandum,  dated  5 August  1970,  subject:  Cost  Growth 

Definitions.  These  definitions  will  be  used  in  documenting  cost  changes 
from  one  cost  estimate  to  a succeeding  estimate. 

2.  Text  of  Memorandum. 

As  Indicated  in  my  memorandum  of  November  26,  1969,  the  views  of  each 
addressee  were  obtained  relative  to  the  tentative  definition  of  "cost 
growth"  which  was  distributed  at  that  time.  These  views  have  been 
carefully  considered  and  changes  made  to  improve  the  clarity  or  the 
categorization  of  the  reasons  for  "cost  growth." 

Distributed  with  this  memorandum  is  the  definition  of  "cost  growth." 

This  will  apply  to  the  net  increased  cost  to  the  Government  of  items 
or  services  procured  or  to  be  procured.  There  are  nine  listed  categories 
of  reasons  for  cost  growth  which  provide  the  visibility  required. 

This  definition  for  "cost  growth"  or  "cost  decrease"  will  be  used  when 
necessary  to  explain  programs,  budgets  or  contracts.  For  internal 
management  purposes,  any  of  the  categories  may  be  grouped  or  further 
stratified  to  serve  management  needs.  However,  any  grouping  of 
categories  thus  used  must  be  capable  of  being  identified  by  the  nine 
individual  categories  if  this  is  later  required  for  reconciliation 
purposes. 

It  is  expected  that  this  "cost  growth"  definition  will  be  used  wherever 
appropriate  in  management  reporting,  testimony,  official  correspondence 
or  speeches,  to  explain  instances  of  cost  growth. 

COST  GROWTH 


Cost  Growth  is  the  net  change  of  an  estimated  or  actual  amount  from  a 
base  figure  previously  established.  The  base  must  be  relatable  to  a 
program,  project  or  contract  and  be  clearly  identified  including 
source,  approval  authority,  specific  items  Included,  specific  assumptions 
made,  date  and  amount.  The  events  causing  "Cost  Growth"  must  then  be 
identified  by  one  or  more  of  the  following  categories  and  the  appropriate 
amount  of  each  shown  as  "estimated"  or  "actual."  These  categories  do 
not  necessarilv  determine  whether  the  cost  growth  could  have  been  avoided 
by  the  Government  or  contractor  or  both.  They  provide  the  essential 
visibility  and  information  required  to  determine  the  cause  of  the  cost 
growth. 
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CATEGORIES 


1.  Engineering  Change  - An  alteration  in  the  physical  or  functional 
characteristics  of  a system  or  item  delivered,  to  be  delivered,  or 
under  development,  after  establishment  of  such  characteristics. 

2.  Quantity  Change  - A change  in  quantity  to  be  procured,  the  cost  of 
which  is  computed  using  the  original  cost-quantity  estimating  relation- 
ships, thereby  excluding  that  portion  of  the  current  price  attributable 
to  changes  in  any  other  category. 

3.  Support  Change  - A change  in  support  item  requirements  (e.g. , spare 
parts,  training,  ancillary  equipment,  warranty  provisions.  Government 
furnished  property/equipment,  testing,  etc.). 

4.  Schedule  Change  - A change  in  a delivery  schedule,  completion  date 
or  intermediate  milestone  of  development  or  production. 

5.  Unpredictable  Change  - A change  caused  by  Acts  of  God,  work  stoppage. 
Federal  or  State  Law  changes  or  other  similar  unforeseeable  events. 
Unforeseeable  events  Include  extraordinary  contractual  actions  under 

the  authority  of  PL  85-804  except  that  formalization  of  informal  com- 
mitments should  be  reflected  under  the  other  categories,  as  appropriate 
and  not  Included  under  this  category. 

6.  Economic  Change  - A change  due  to  the  operation  of  one  or  more 
factors  of  the  economy.  This  Includes  specific  contract  changes  related 
to  economic  escalation  and  the  economic  impact  portion  of  contract 
quantity  changes  computed  using  the  original  contract  cost-quantity 
relationship.  This  also  includes  changing  real  dollar  amounts  in 
program  estimates  to  reflect  (1)  revised  economic  Impact  or  (2)  defini- 
tized  contract  amounts. 

7.  Estimating  Change  - A change  in  program  or  project  cost  due  to 
refinements  of  the  base  estimate.  These  include  mathematical  or  other 
errors  in  estimating,  changing  the  base  year  of  the  constant  dollars, 
revised  estimating  relationships,  changing  from  constant  dollars  to 
real  dollars,  etc. 

8.  Contract  Performance  Incentives  - A net  change  in  contractual  amount 
due  to  the  contractor's  actual  performance  being  different  than  was 
predicted  by  performance  (including  delivery)  incentive  targets;  as 
differentiated  from  cost  incentive  targets;  established  in  an  FPI  of 
CPIF  contract.  This  category  also  includes  any  changes  in  amounts 

paid  or  to  be  paid  a contractor  due  to  (1)  award  fee  for  performance 
accomplishments  under  a cost  plus  award  fee  contract  or  (2)  the  sharing 
provisions  of  a value  engineering  incentive  clause  included  in  any 
type  of  contract. 


9.  Contract  Cost  Overrun  (Underrun)  - A net  change  in  contractual 
amount  over  (under)  that  contemplated  by  a contract  target  price  (FPI 
contract) , estimated  cost  plus  fee  (any  type  cost  reimbursement 
contract)  or  redeterminable  price  (FPR  contract),  due  to  the  contractor's 
actual  contract  costs  being  over  (under)  target  or  anticipated  contract 
costs,  but  not  attributable  to  any  other  cause  of  cost  growth  previously 
defined.  Offsetting  profit  or  fee  adjustments  attributable  to  cost 
incentive  provisions,  if  any,  shall  be  considered  in  determining  the 
net  contract  cost  overrun  (underrun) . 
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